A Method And An Apparatus For Reordering A Decoded Picture Sequence 

Using Virtual Picture 

Field of Invention 

The present invention is related to video display systems, and more particularly, to the 
reordering of decode order into display order in video display systems. 

Background of Invention 

The CCITT/ISO committee has standardized a set of compression and decompression 
algorithms for still and motion digital video. These standards, including the JPEG, 
MPEG and H.261 compression schemes, are commonly applied in video conferencing, 
CD-ROM or DVD-ROM based interactive videos for education and entertainment, 
video, video on demand (VOD) applications, satellite video transmission applications 
and many other applications which require communication of motion digital video. 
The MPEG standard was drafted by the Moving Picture Coding Experts Group 
(MPEG) which operates within the framework of the Joint ISO/IEC Technical 
Committee (JCCI) on Information Technology. The draft provided a standard for 
coded representation of moving pictures, audio and their combination. 
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In General, motion estimation is a technique for compressing data in image 
processing method by estimating a motion vector. The motions within the pixels of 
current frame (or fields) and pixels of previous frame, in successive image signals, are 
transmitted as a motion vector and a difference of data between macro-blocks of the 
previous frame and the current frame. 

In motion estimation, frames of input image signals are divided into I-pictures (Intra- 
coded picture), P-pictures (Predictive-coded picture), and B-pictures (Bi-directionally 
predicted-coded picture). The I-pictures are frames (or fields) which are coded 
without using motion estimation. The P-pictures are frames that perform only the 
forward motion estimation by using I-pictures or other P-pictures as references. The 
B-pictures are frames that perform the forward and backward estimation by using I- 
pictures or P-pictures as references. In standard MPEG 2, the so-called Group of 
Picture is formed from a series of image frames comprising of I-pictures, P-pictures 
and B pictures, such asI-B^B^P-B^B*P>B^B^P-B-B - the pictures 
are coded and transmitted to the display system for display. 



The input coded (or compressed) sequence has to be decoded into decoding sequence 
(i.e. decode order), the order of the decode sequence is not the same as the order of 

4MTK200101US, MTK1-01-002 2 



display sequence (i.e. display order), hence the decode order has to be re-ordered into 
display order for correct display. In a common display and decode control flow, 
reordering is done by storing pictures of decode order into different decoding buffer, 
then the display system retrieves the pictures stored in different buffers according to 
the display order. 

But under some circumstances, it is necessary to display pictures stored in one of the 
decoding buffer continuously, therefore the above-mentioned reordering method is 
not applicable. For example, the digest view function provides users with the ability 
to quickly preview the content of the disc(s), it is implemented by displaying 
simultaneously the scaled down version of previously-decoded picture with the 
current decoding picture in the same sequence (/or different sequences). In order to 
support the preview function, current decoding picture and previous-decoded picture 
must be stored in the one single buffer for the video output module to display the 
content in the display buffer continuously. Furthermore, current decoding pictures 
must be stored into the display buffer in accordance with the display order, which is 
illustrated in Table. 1. 
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Table. 1 decode order and display order 
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Assume the size of all buffers are the same, and the display system can scale down the 
decoded picture by a given scale factor such as one third or one fourth. In Table. 1 , 
after 10 is decoded and stored in one decoded buffer at time Tl, P3 should be decoded 
and the scaled down 10 should be stored into the display buffer at time T2. When the 
display system finishes decoding Bl and B2, the scaled down version of P3 should be 
stored into display buffer thereafter During the decoding and displaying process, 
the I-pictures and the P-pictures, which are used as reference pictures, will be stored 
twice, that is one normal size into decode buffer and the other scaled down size into 
display buffer. In the meantime, B-pictures will be decoded and sent to display 
buffer directly. 

Conventional technique tackles the problem by decoding the reference pictures twice, 
one stored in the decoded buffer in normal size and the other stored in display buffer 
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in scaled down size. However, it will require the system to retrieve the picture again 
for second decoding, which makes the decoding control scheme complicated. 
Another conventional technique provides additional DMA hardware with scaling 
function to move the pictures stored in decoded buffer to display buffer, but it requires 
additional hardware cost. Still another conventional technique uses system RISC to 
read pictures stored in the decoded buffer and write them into display buffer, the price 
is to cost excessive RISC I/O cycles and bring down RISC performance. 

To reorder the decode order into display order, the system has to either decode the 
reference pictures twice, or add additional DMA hardware, or increase I/O cycles of 
system RISC, and sometimes the system has to contiguously display the pictures 
stored in one buffer. The present invention discloses a method for reordering a 
decode order into a display order by inserting a virtual picture, therefore minimum 
hardware and software cost can be achieved. 

Summary of Invention 

The present invention discloses a method and a circuit for reordering a decode order 
into a display order of images by inserting a virtual picture in the decode order, the 
decode order comprises an I-picture, a P-picture, and a B-picture. Upon receiving a 
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compressed picture sequence, the present invention performs the following steps: (1) 
determining a first picture of the compressed picture sequence is an I-picture, a P- 
picture, or a B-picture; (2) if the first picture is I-picture, decoding the first picture and 
storing the decoded first picture into a first buffer; and (3) obtaining a first virtual 
picture according to a predetermined manner, sending the first virtual picture to a 
second buffer for display. 

Description of Drawings 

Fig. 1(a) and 1(b) schematically illustrate the circuit of present invention. 
Fig.2(a), 2(b), 3(a), and 3(b) are the flow diagrams of present invention. 
Fig.4 schematically illustrates one preferred embodiment of present invention. 



Detailed Description of Present Invention 

Please refer to Fig. 1(a), the present invention discloses an apparatus for reordering a 
decode order into a display order, comprises a parameter generator 100 for generating 
a parameter 105; a motion compensator 110, in response to the parameter 105, for 
outputting a virtual picture 115 using the first decode picture stored in the first buffer 
131 ; a bus 120 for connecting every component in the present invention. 
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Please refer to Fig. 1(b) for detailed drawing of the memory 130, the memory 130 
comprises a first buffer 131 for storing a first decoded picture of the decode order, a 
second buffer 132 for storing the virtual picture, and a third buffer 133 for storing a 
second decoded picture of the decode order. 

Continuing with Fig. 1(a), the present invention comprises a display controller 140 
connecting to the second buffer 132 for displaying the virtual picture 115. All the 
pictures are transmitted via bus 120. The display controller 140 obtains the virtual 
picture 115 and the original decoded pictures, performs OSD, subpicture, highlight 
mixing function and outputs pictures to video display machine such as TV. 

It is noted that pictures in the compressed picture sequence are either frame or field 

type, virtual pictures are also frame or field type, the corresponding parameter 105 

( > 

comprising\in MPEG2 format): 

a. the picture coding type (picture_coding_type) of virtual picture is P-picture, 
every macro-block in a virtual picture is a skipped macro-block; 

b. the prediction of the macro-blocks in a virtual frame picture should be made as 
if frame_motion_type is "Frame-based"; 
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c. the prediction of the macro-blocks in a virtual field picture should be made as 
if field_motion_type is "Field-based"; 

d. motion vector is user-defined, preferably 0; 

e. coded-block pattern is 0; 

f. as to virtual field picture, the prediction should be made from the field of the 
same parity. 

Preferably, the parameter 105 also comprises a scale factor, so the width and height of 
a virtual picture can be different from the original picture. 

In the following descriptions and illustrations, blocks having the same number mean 
the same operation. 

In one aspect of the present invention, please refer to Fig.2(a), step 200 obtains a 
compressed picture sequence, step 205 determines a first picture in the compressed 
picture sequence is an I-picture, a B-picture or a P-picture, if the first picture is I- 
picture, step 210 decodes the first picture and stores the decoded first picture into a 
first buffer 131. Step 215 uses a decoded picture pre-stored in a third buffer 133, 
responsive to a parameter 105, generates the first virtual picturellS and send it to a 
second buffer 132 for display. 
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The present invention further performs the following steps: step 220 determines a 
second picture, if the second picture is P-picture 3 step 222 decodes the second picture 
and stores the decoded second picture into a third buffer 133. Step 223 uses a 
decoded picture pre-stored in the first buffer 131, responsive to a parameter 105, 
generates the second virtual picture and send it to the second buffer 132 for display. 

In step 220, if the second picture is I-picture, step 225 decodes the second picture and 
stores the decoded second picture into the third buffer 133. Step 226 uses a decoded 
picture pre-stored in the first buffer 131, responsive to the parameter 105, generates 
the second virtual picture and send it to the second buffer 132 for display. 

In step 205, if the first picture is P-picture as shown in Fig.2(b), the rest follows the 
case in which the first picture is I-picture. 

Besides, it should be noted that in Fig.2(a), the step 210 and 215 are interchangeable, 
which means the present invention can generate a virtual picture 115 first then 
decodes the first picture, shown as step 305 and 310 of Fig.3(a). The rest is the same 
as Fig.2(a). Also in Fig.2(a), after step 220 is done, the following step 222 and step 
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223 are interchangeable, step 225 and step 226 are interchangeable. The same can 
be applied in Fig.3a, wherein step 222 and 223 are interchangeable, step 225 and step 
226 are interchangeable. 

While in Fig.2(b), step 210 and step 215 are interchangeable, shown as step 305 and 
310 of Fig.3(b). And the step 222 and 223 in Fig.2(b) are interchangeable, step 225 
and step 226 are interchangeable. In the same way, step 222 and step 223 of Fig.3(b) 
are interchangeable, whereas step 225 and step 226 in Fig.3(b) are interchangeable. 

Please refer to Fig.4, in one preferred embodiment of the present invention, upon 
receiving picture P6, decode picture P6 and store the decoded picture P6 into first 
buffer 131. In response to the parameter 105, generate a virtual picture using P3 
stored in the third buffer 133, send the virtual picture to the second buffer 132. 
Subsequently, decode the received B4 and B5, send the decoded B4 and B5 directly to 
the second buffer 132, the display controller 140 will use the pictures stored in the 
second buffer 132 for display. The next picture received is 19, decode 19 and store 
the decoded 19 in the third buffer 133, also in response to the parameter 105, generate 
a virtual picture using P6 stored in the third buffer 131, send the virtual picture to the 
second buffer 132. The following B7 and B8 will be decoded, and the decoded B7 
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and B8 will be sent to the second buffer 132 directly, the display controller 140 uses 
the pictures stored in the second buffer 132 for display thereafter. 

Continuing with Fig.4, when P12 is received, decode picture P12 and store the 
decoded picture P12 into first buffer 131. In response to the parameter 105, generate 
a virtual picture using 19 stored in the third buffer 133, send the virtual picture to the 
second buffer 132. Subsequently, decode the received BIO and Bll, send the 
decoded BIO and Bll directly to the second buffer 132, the display controller 140 
will use the pictures stored in the second buffer 132 for display. The next picture 
received is PI 5, decode PI 5 and store the decoded PI 5 in the third buffer 133, then in 
response to the parameter 105, generate a virtual picture using PI 2 stored in the third 
buffer 131, send the virtual picture to the second buffer 132. 

Please refer to Table.2, which illustrates how to use virtual pictures to reorder the 
decode order and the display order. 
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Table.2 decode order and display order (with virtual pictures inserted) 
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While in Table.2, the motion compensator 110 is treated as a special DMA channel by 
moving data directly between different buffers to create virtual picture. Besides, it 
should be noted that whenever the present invention starts to decode a reference 
picture (I-picture or P-picture) 5 the present invention will stored the received picture 
in the first or the third buffer and override the older data stored in the first or the third 
buffer. Please refer to Fig.4, when P6 is received, it will be stored in the buffer 
where originally 10 is stored, not the buffer where P3 is stored. 
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The present invention uses the motion compensator 110 to generate the virtual picture, 
all operations can be completed by motion compensator 110 itself. Thus avoiding 
the need for additional DMA hardware, or increasing I/O cycles of the RISC, and the 
complexity and the cost of hardware and software design can be reduced. 

It will be apparent to those skilled in the art that various modifications and variations 
can be made in the present invention without departing from the spirit or scope of the 
invention. Thus, it is intended that the present invention covers the modification and 
variations of this invention provided they come within the scope of the appended 
claims and their equivalents. 
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